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^College  students  were  given  a complex  design  problem:  to 
design  a restaurant  in  a building  that  used  to  be  a church. 

A diversity  of  background  information-sources  and  problem 
constraints  were  given,  mirroring  the  situation  for  software 
design.  The  study  was  also  concerned  with  the  effects  on 
the  design  solution  of  gradations  in  the  modifiability  of 
pre-existing  structures  (ranging  from  exterior  walls  of  the.. 
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building  to  furniture  and  architectural  detail) . After 
completing  the  design,  using  a format  of  their  choice, 
participants  filled  out  a questionnaire  concerning  their 
strategy,  goals  and  information  sources.  The  primary 
difficulty  in  evaluating  these  designs  is  the  same  as  in 
software  design:  the  solution  is  essentially  conceptual  and 
cannot  be  directly  tested.  Although  a common 
design-evaluation  method  is  peer-ratings,  in  the  current 
study  we  developed  two  objective  methods.  Practicality  was 
measured  as  the  ratio  of  functional  requirements  satisfied 
in  the  design  to  all  requirements  (the  latter  derived  from 
an  a priori  elaboration  of  design  goals) . Originality  was 
based  on  an  information  analysis  of  the  fejatures  of  each 
person's  design  relative  to  all  designs.  ljfithin  groups 
having  similar  SAT  scores,  there  was  a negative  correlation 
between  originality  and  practicality.  Both  measures  were 
partially  predicted  by  the  participant's  expressed  goals  but 
not  by  expressed  strategy  or  stylistic  differences  between 
participants.  A word  list  found  to  improve  design  on 
simpler  problems  in  an  earlier  experiment  was  given  half  the 
participants,  who  subsequently  had  significantly  higher 
practicality  scores  (no  effect  on  originality  was  found) . 
Modifications  to  existing  structures  differed  but  still 
formed  a perfect  linear  ordering:  no  participant  modified 
any  higher  level  structure  without  also  modifying  lower 
level  structures.  In  general,  participants  tended  to  design 
in  a top-down,  planned  approach.  The  results  of  this 
experiment  produced  a detailed  picture  of  structural  design 
which  seems  very  similar  to  software  design. 
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Abstract 


Twenty-nine  college  students  were  given  a complex  design  problem:  to  design  a restaurant  in  a 
building  that  used  to  be  a church.  Participants  were  given  extensive  background  data  on  the 
expected  clientele  for  the  restaurant,  the  cost  and  spatial  requirements  for  various  items,  the 
floor  plan  of  the  church,  site  information,  equipment  costs,  remodeling  costs,  and  typical 
customer  complaints  about  similar  restaurants.  Such  a diversity  of  information-sources  and 
constraints  mirrors  the  situation  for  software  design.  The  study  was  also  concerned  with  the 
effects  on  the  design  solution  of  gradations  in  the  modifiability  of  pre-existing  structures 
ranging  in  this  case  from  exterior  walls  of  the  building  to  internal  stuctural  walls,  to  non- 
structural  walls,  to  movable  partitions,  to  furniture  and  architectural  detail.  Participants  were 
free  to  present  their  ideas  for  the  design  in  any  format.  After  completing  the  design,  partici- 
pants were  asked  to  fill  out  a questionnaire  concerning  their  strategy,  goals,  and  information 
sources.  The  primary  difficulty  in  evaluating  these  designs  is  the  same  as  in  software  design; 
viz.,  that  the  solution  is  essentially  conceptual.  It  cannot  be  directly  tested.  Although  one 
common  method  of  evaluating.4e$ign  in  such  situations  is  to  have  them  rated  by  peers,  in  the 
current  study  we  attempted  Ip  develop  more  objective  methods.  In  particular,  an  a priori 
analysis  of  the  design  goals  resulted  in  an  expanded  tree  of  possible  functional  requirements. 
Practicality  was  measured  as  tfife- ratio  of  functional  requirements  satisfied  to  all  requirements. 
Originality  was  based  on  an  information  analysis  of  the  features  of  each  person’s  design 
relative  to  all  designs.  Within  each  grouping  of  participants  with  similar  SAT  scores,  there  was 
a negative  correlation  between  originality  and  practicality.  Originality  and  practicality  were 
both  partially  predicted  by  the  participant’s  expressed  goals.  The  measures  were  uncorrelated 
with  any  expressed  strategy  variables,  or  with  any  noted  stylistic  differences  between  partici- 
pants. Half  of  the  participants  were  given  a word  list  that  had  been  found  to  improve  design 
on  two  simpler  problems  in  an  earlier  experiment.  Participants  given  this  unstructured  aid  had 
significantly  higher  practicality  scores  but  no  effect  on  originality  was  found.  The  original 
floor  plan  of  the  church  contained  various  structures  which  might  or  might  not  be  changed  by 
the  participants.  Participants  differed  in  the  degree  to  which  they  modified  the  existing 
structures.  However,  the  structures  that  participants  chose  to  modify  formed  a perfect  linear 
ordering:  no  participant  modified  any  higher  level  structure  without  also  modifying  lower  level 
structures.  Strategy  differences  between  participants  and  implications  of  the  findings  for  design 
aids  are  discussed.  Generally,  participants  tended  to  design  in  a top-down,  planned  approach. 
The  results  of  this  experiment  produced  a detailed  picture  of  structural  design  which  seems 
very  similar  to  software  design. 
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An  Experimental  Investigation  of  the  Design  Process 
1.  Introduction 

I . I Importance  of  Design 

The  New  York  Times  (12/26/76;  p.34),  contained  the  following  item:  "Design  flaws  were 
responsible  for  about  two-thirds  of  the  52  million  cars  and  trucks  recalled  in  the  last  10  years 
to  correct  safety  defects.  Federal  safety  officials  say....  Recall  campaigns  not  only  are  bad  for 
public  relations  but  they  are  also  costly.  The  postage  alone  for  certified  letters  to  the  most 
recent  770,000  owners  of  recalled  vehicles  cost  General  Motors  and  Ford  about  $400,000". 


Design  errors  and  their  tremendous  attendant  costs  are  not  limited  to  the  automobile 
industry.  In  the  field  of  software,  for  example,  the  cost  of  design  is  particularly  important.  By 
1985,  it  has  been  estimated  that  software  costs  will  constitute  90  per  cent  of  total  Data 
Processing  system  costs  (Boehm,  1973).  Design  usually  is  estimated  to  account  for  about 
one-third  of  the  time  and  cost  of  producing  software  (Wolverton,  1974).  However,  testing 
usually  accounts  for  nearly  half  of  the  man-hours  involved  in  software  development,  and  about 
half  to  two-thirds  of  the  errors  in  code  originate  during  the  design  phase.  Thus,  improvements 
in  software  design  could  considerably  shorten  the  development  process,  lower  costs,  and 
improve  reliability. 

The  literature  on  software  design  is  generally  devoid  of  empirical  work.  Though  the 
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numerous  heuristics  for  software  design  are  useful  in  their  own  right,  (e.g.,  Dahl,  Dijkstra,  & 
Hoare,  1972;  Jackson,  1975;  Yourdon,  1975)  they  do  not  explicate  the  cognitive  processes 
operating  in  complex  design  problem  solving.  For  its  part,  cognitive  psychology  has  tended  to 
focus  on  the  analysis  of  simple,  puzzle-like  problems  solved  in  short  time  periods  by  college 
students  (e.g.,  Thomas,  1974).  Some  attempts  exist  to  extend  the  theoretical  work  on  simple, 
well-formulated  problem  solving  to  design  problems  (Reitman,  1965;  Simon,  1973;  Eastman, 


1969;  Freeman  and  Newell,  1971).  Though  empirical  studies  of  design  problems  are  rare,  it 
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would  seem  that  the  study  of  applied  psychology  in  this  area  could  provide  substantial  benefits. 
The  current  study  was  designed  to  explore  four  issues  in  design:  (1)  the  relationship  of  goals  to 
the  design  product,  (2)  the  production  of  design  ideas,  (3)  the  objective  measurement  of 
creativity  and  (4)  the  influence  of  strategy  on  performance. 

1.2  Goals. 

An  excellent  empirical  case  study  by  Mitroff  (1972)  illustrates  the  interactions  of 
technical  and  social  variables  in  a real-world  design  situation.  An  important  finding  of  this 
study  is  that  the  client  who  calls  for  a design  may  either  over-constrain  the  designer  with 
requirements,  making  a design  that  fits  ALL  the  requirements  impossible,  or  under-constrain 
the  designer  so  that  a vast  space  of  potential  designs  exists,  probably  only  some  of  which  will 
really  satisfy  the  client’s  goals.  The  latter  outcome  will  occur  because  the  client  may  not  be 
able  to  explicitly  state  all  his  goals. 

Work  on  software  design  (Scott  and  Simmons,  1973;  Walston  and  Felix,  1977)  suggests 
also  that  communication  problems  between  the  client  and  the  designer  are  frequent  causes  of 
design  difficulty.  One  of  the  interests  in  the  current  experiment  was  the  extent  to  which 
people  would  be  aware  of  their  design  goals  and  the  extent  to  which  these  goals  would 
influence  the  design.  If  a better  understanding  of  design  goals  and  their  relation  to  design 
products  could  be  established,  improved  tools  for  communicating  design  goals  might  be 
established  with  resulting  improvement  in  the  design  products.  Thus  far,  the  problems  of 
communicating  about  purpose  have  been  largely  ignored  in  the  software  literature  (But  see 
Douglas,  1977). 

1.3  The  production  of  design  ideas. 


A second  difficulty  in  design  is  finding  the  proper  design  elements  (Malhotra,  Thomas, 


Carroll,  and  Miller,  1978).  In  some  cases,  people  have  relevant  information  in  long  term 


memory  but  the  information  may  not  spontaneously  present  itself  to  consciousness.  Apparent 
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ly,  information  is  only  readily  retrievable  in  a limited  number  of  ways  (Friendly,  1977)  and  it 
is  unlikely  that  people  can  forsee  all  the  possible  uses  to  which  information  could  be  usefully 
put.  Hence,  they  will  later  be  able  to  retrieve  the  information  only  in  a few  of  the  potentially 
useful  ways.  A possible  way  to  help  the  participant  retrieve  some  of  this  information  is  to  cue 
the  participant  with  checklists,  images,  or,  as  in  the  present  study,  with  words.  Previous  work 
(Thomas,  Lyon,  and  Miller,  1977)  indicated  that  an  unstructured  design  aid  consisting  of  an 
unorganized  list  of  diverse,  highly  concrete  words  improved  the  design  performance  of  college 
students  on  two  simple  design  problems.  One  purpose  of  the  present  experiment  was  to  extend 
[ this  finding  to  a more  complex  design  problem. 


1.4  Product  measures:  originality  and  practicality. 


Another  difficulty  in  attempting  to  extend  experimental  results  on  solving  simple  problems 
to  design  problem  solving  is  the  appropriate  choice  of  dependent  measures  of  performance. 
Creativity  is  generally  defined  to  imply  both  originality  and  practicality.  For  example.  Bourne, 
Eckstrand,  and  Dominowski  (1971)  say  "For  behavior  to  be  termed  creative,  it  must  be 
original  (statistically  infrequent),  and  in  addition,  a problem  must  be  solved  or  some  other 
criteria  of  relevance  or  practicality  must  be  met."  The  present  study  defines  more  objective 
measures  of  both  originality  and  practicality. 

1.5  Process  measures:  the  strategies  of  participants. 

In  the  study  of  software  particularly,  but  also  in  books  on  design  in  general,  it  is  often 
assumed  that  the  designer’s  effectiveness  may  be  greatly  enhanced  by  impacting  his  strategy 
. (See,  e.g.,  Jones,  1969  for  an  overall  design  strategy,  as  well  as  for  a review  of  particular 

sub-strategies).  We  therefore  assessed  the  strategies  of  participants  by  a post-design  question- 
i naire  as  well  as  by  characterizing  stylistic  features  of  the  design  product. 
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2.  Method 


In  this  experiment,  an  attempt  was  made  to  study  as  much  of  the  design  process  as  would 
be  feasible  in  a single  experimental  session.  To  accomplish  this,  the  particular  design  situation 
chosen  was  one  for  which  college  students  have  some  relevant  background  information  and 
some  motivation  to  do  well.  The  design  of  a restaurant  fit  these  criteria.  For  versimilitude,  a 
case  study  (Reagen  & Stevens,  1972)  of  a Bonanza  Steak  House  was  used.  In  addition  to 
material  from  the  case  study,  numerous  other  information  sources  were  provided  to  the 
participants. 


Participants  were  told  that  the  restaurant  would  be  placed  in  an  old  church.  They  were 
provided  with  a diagram  of  the  existing  layout  of  the  church.  (See  Figure  1).  In  many  design 
situations,  one  must  modify  an  existing  procedure  or  structure.  The  existing  church  layout  was 
designed  by  the  experimenters  to  include  a number  of  features  of  varying  permanence.  This 
was  to  allow  us  to  test  the  variability  between  participants  in  how  much  they  were  willing  to 
change  the  existing  structure.  The  church  layout  contained  external  walls,  internal  structural 
walls,  internal  non-structural  walls,  internal  movable  partitions,  and  architectural  details  and 
furniture.  It  was  hypothesized  that  these  would  form  a hierarchy  of  modifiability. 


Insert  Figure  1 About  Here 


2.1  Participants 


Twenty-nine  paid  volunteers  from  a small,  nearby  liberal  arts  college  served  as  partici- 
pants. These  participants  were  generally  above  average  in  terms  of  their  SAT  scores. 
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2.2  Procedure 

Participants  were  run  as  a group.  Each  participant  was  initially  furnished  with  a folder 
containing  the  printed  information  sources  (See  section  2.3).  Each  participant  was  told  that 
the  task  was  to  design  a restaurant  and  that  the  folder  contained  potentially  relevant  informa- 

» 

tion.  They  were  told  to  read  this  information  and,  when  they  felt  ready,  to  ask  for  additional 
materials  in  order  to  communicate  their  design.  Participants  were  told  that  they  could  spend  as 
long  as  they  liked  reading  the  material  and  as  long  as  they  liked  designing  the  restaurant.  They 
were  also  told  that  they  were  free  to  communicate  their  ideas  about  the  restaurant  in  any 
format  that  they  wished  including  pictures,  floor  plans,  and  written  comments.  Participants 
were  also  informed  that  they  were  welcome  to  take  notes  and  that  they  would  be  allowed  to 
refer  to  their  own  notes  as  well  as  the  printed  information  during  the  design.  The  participants 
were  requested  to  work  alone.  When  a participant  indicated  that  he  or  she  was  ready  to  begin 
designing,  the  experimenter  noted  the  time  and  gave  the  participant  a variety  of  types  of  paper 
(lined,  plain,  quad),  a ruler,  and  several  pencils  of  various  colors.  When  a participant  complet- 
ed designing,  a questionnaire  was  given  the  participant.  The  questionnaire  asked  for  back- 
ground information  on  the  participant  including  SAT  scores,  and  asked  information  about  the 
participant’s  goals  during  design,  the  information  sources  used,  and  the  strategy  that  the 
participant  had  adopted. 

2.3  Information  sources 

A 

The  information  sources  given  to  the  participant  included  four  pages  of  typewritten 
background  material  on  the  owner  of  the  restaurant,  the  Bonanza  chain  and  the  location  and 
financing  of  the  restaurant.  In  addition,  participants  were  given  a sheet  of  site  information,  a 
list  of  typical  costs  for  various  types  of  renovation,  a list  of  costs  for  restaurant  equipment,  a 
floor  plan  of  the  current  church,  a written  verbal  description  of  the  church,  a list  of  common 
complaints  of  restaurant  customers,  a list  of  the  spatial  requirements  for  various  aspects  of 
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2.4  Scoring  procedures 


A creative  response  is  generally  defined  to  be  one  that  is  both  original  and  workable.  In 
this  study,  an  attempt  was  made  to  develop  a universal  method  of  defining  originality  and 
practicality.  To  this  end,  a mathematical  definition  of  originality  O was  calculated  according 
to  the  following  procedure.  First,  a feature  analysis  was  performed  on  the  restaurant  designs. 
Every  feature  found  in  any  design  was  listed.  Second,  every  individual  design  was  checked  for 
each  feature.  The  presence  of  a feature  was  coded  as  1 and  its  absence  by  a 0.  Third,  by 
summing  over  participants,  the  probability  of  the  presence  or  absence  of  each  feature  was 
calculated.  Fourth,  the  information  H(i)  in  each  design  was  calculated  (taking  the  sum  of 
probabilities  times  the  log  to  the  base  two  of  the  reciprocal  of  the  probabilities).  Fifth,  we 
calculated  the  information  present  in  the  most  original  possible  design,  given  the  distribution  of 
features  in  the  actual  data  H(max).  (This  was  done  quite  simply.  For  each  feature,  a 
hypothetical,  optimally  original  design  had  a feature  if  it  was  more  commonly  left  out,  and 
omitted  a feature  if  it  was  more  commonly  included).  Finally,  each  participant  was  assigned  a 
ratio  value  by  dividing  the  information  in  his  or  her  design  by  the  maximum  possible  informa- 
tion present  in  a design,  given  the  data:  O (i)=  H (i)  / H (max).  This  measure  of  originality 
has  the  theoretical  advantages  that  it  is  closely  related  to  information,  it  has  an  objective  zero 


point,  and  exhibits  other  desirable  properties  (e.g.,  transitivity,  and  reflexivity). 
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A similar  measure  ranging  from  0 to  1 was  defined  for  practicality.  A thorough  analysis 
of  the  goals  for  the  restaurant  layout  was  made  on  an  a priori  basis.  Goals  were  refined  to 
functional  requirements.  That  is,  higher  level  goals  were  broken  down  into  successively  mort 
measurable,  more  specific  subgoals.  This  resulted  in  a hierarchy  of  goals.  Each  participant’s 
design  was  examined  from  the  standpoint  of  each  goal  at  each  level  of  the  hierarchy.  If  the 
participant’s  design  was  judged  to  have  met  that  goal,  it  was  assigned  a 1;  if  not,  a 0.  Finally, 
each  participant’s  practicality  score  P,  was  calculated  to  be  the  ratio  of  goals  and  subgoals 
satisfied  to  the  number  of  goals  in  the  entire  goal  hierarchy.  (The  goal  tree  is  shown  in  Figure 
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2).  In  the  present  case,  since  participants  were  free  to  make  their  own  decisions  regarding  the 
relative  importance  of  various  goals,  weightings  were  not  assigned  to  different  goals.  For  some 
situations,  a more  appropriate  variant  could  be  developed  which  would  include  a weighting 
function. 


Insert  Figure  2 About  Here 


It  was  judged  that  two  of  the  twenty-nine  participants  had  not  really  accepted  the  basic 
goal  of  building  a usable  restaurant  in  the  church  and  their  designs  were  not  given  a value  for 
P and  were  excluded  from  the  analyses  described  below. 

3.  Results 

3.1  Originality  and  Practicality. 

On  the  questionnaire,  each  participant  was  asked  to  rate  how  important  it  was  that  the 
design  was  original,  novel,  complex,  practical,  workable,  and  imaginative.  Participants  tended 
to  split  into  two  groups:  those  whose  emphasis  was  on  being  practical  and  workable  versus 
those  who  were  concerned  with  originality,  novelty,  and  imaginativeness.  The  ratings  of 
workability  and  practicality  were  positively  correlated  with  each  other  and  negatively  correlat- 
ed with  the  ratings  of  imaginativeness,  novelty,  and  originality,  which  in  turn  were  all  positive- 
ly intercorrelated. 
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Objective  originality  scores,  as  defined  above,  varied  from  .624  to  .766  (See  Figure  3). 
Objective  originality  (O)  was  not  correlated  with  the  objective  measure  of  practicality  (P) 
overall  (r«.02).  However,  when  participants  were  divided  according  to  SAT  scores  into  six 
categories,  there  was  a negative  correlation  between  O and  P within  each  group.  Although 
there  were  too  few  participants  in  each  group  for  any  of  the  six  correlations  to  reach  signifi- 
cance, a two-tailed  sign  test  would  make  the  overall  result  marginally  significant  (p=.03). 


Insert  Figure  3 About  Here 


Aside  from  this  result,  the  only  measure  at  all  predictive  of  O was  the  participant’s 
avowed  concern  with  having  a design  that  was  workable  and  practical.  The  correlation 
between  the  sums  of  these  measures  and  O was  -.452,  (F(  1,18)  =4.75,  p<.05).  Participants 
who  expressed  high  concern  with  being  practical  scored  low  on  this  objective  measure  of 
originality.  The  participant’s  other  statements  about  strategy  and  SAT  scores  were  both 
non-informative  with  respect  to  O. 

Participants  varied  in  the  proportion  of  functions  fulfilled  (P)  from  .34  to  .71.  (See 
Figure  4).  Objective  P was  unrelated  to  any  measure  of  the  participant’s  strategy  or  style. 
However,  again,  there  was  a significant  relation  to  the  sum  of  the  participant’s  own  ratings  of 
the  importance  of  originality,  imagination,  and  novelty.  Those  participants  who  expressed  high 
concern  with  these  characteristics  had  designs  that  scored  low  on  P. 


Insert  Figure  4 About  Here 


3.2  An  aid  to  idea  production. 


i 


Half  of  the  participants  were  given  an  extensive  word  list  but  given  no  instructions  to  use 
it.  Those  participants  who  received  the  list  scored  significantly  higher  on  P than  those 
participants  who  did  not  use  the  list.  By  looking  at  whether  or  not  the  participant  was  given 
this  aid,  at  SAT  score,  and  at  expressed  goals,  a multiple  correlation  of  .84  in  predicting  P was 
obtained  (F(3,16)=12.85,p<.01). 


i 


Experimental  Investigation  of  Design 


Page  12 


An  earlier  study  (Thomas,  Lyon,  and  Miller  1977)  had  shown  this  aid  to  be  effective  in  a 
simpler  structural  and  a procedural  design  problem.  The  effectiveness  of  the  aid  was  surprising 
in  the  current  experiment  however,  since  the  participants  were  not  required  to  use  the  aid  and 
the  result  should  therefore  be  replicated. 

3.3  The  Participant’s  Approach 

Strategy.  An  issue  of  particular  interest  in  software  design  is  the  strategy  that  a software 
designer  uses.  Specifically,  it  has  often  been  argued  that  top-down  design  is  advantageous 
(e.g.,  Kernighan  & Plauer,  1976)  but  presumably  people  are  not  likely  to  spontaneously  adopt 
this  strategy.  It  was  generally  difficult  to  determine  the  participant’s  strategy  from  the  design 
plans.  Several  questions  on  the  post-design  questionnaire  were  designed  to  assess  the  extent  to 
which  the  participants  felt  that  they  had  used  a top-down  strategy.  Somewhat  surprisingly,  19 
of  the  29  participants  claimed  essentially  to  have  used  a top-down  approach  to  designing  the 
restaurant.  Only  four  claimed  to  have  used  bottom-up  design  and  six  claimed  to  have  used 
some  mixture  of  the  two.  In  view  of  the  admonitions  to  use  top-down  methods  in  program- 
ming, it  seems  odd  that  the  majority  of  untrained  college  students  would  claim  to  spontaneous- 
ly use  a top-down  approach  in  their  design. 

There  are  several  possibilities  for  explaining  this  apparent  discrepancy.  First,  it  is 
conceivable  that  participants  were  simply  incorrect  in  their  self-report  behavior.  Only  those 
aspects  of  their  behavior  that  fit  into  an  easily  coded  organizational  pattern  may  have  been 
recalled.  However,  more  careful  observation  of  some  pilot  participants  as  well  as  clues  from 
the  design  plans  themselves  make  this  alternative  unlikely.  Second,  it  is  possible  that  top-down 
design  is  fairly  natural  in  the  design  of  a structure  but  more  difficult  and  non-intuitive  in  the 
case  of  designing  a procedure  (like  programming).  Third,  perhaps  programming  top-down  is 
not  so  rare  as  its  proponents  would  have  us  believe.  Fourth,  participants  may  have  perceived 
an  organized  approach  to  be  more  highly  esteemed. 
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Another  question  of  interest  was  how  participants  would  change  their  behavior  if  they 
were  to  do  the  design  task  again.  In  response  to  such  a question  there  were  several  categories 
of  answers.  Some  participants  essentially  said  that  they  would  adopt  a different  strategy  (4), 
some  said  that  they  would  change  the  relative  values  of  certain  goals  (11),  while  others  said 
they  would  change  specific  features  of  the  design  (2).  Six  participants  said  they  would  not 
change  anything  while  four  gave  no  answer  and  three  gave  various  miscellaneous  answers. 
Among  those  who  said  they  would  change  their  strategy,  two  said  they  would  be  more  logical 
and  top-down,  one  said  the  goals  should  be  more  explicit,  and  one  said  the  new  approach 
would  be  "more  dimensional."  Among  those  who  said  that  they  would  change  their  goals, 
seven  said  that  they  would  be  more  concerned  about  money  and  three  said  that  they  would  be 
less  concerned  about  the  existing  church  layout  while  one  said  they  would  keep  more  closely  to 
the  initial  layout. 


Apparently  then,  the  changes  that  participants  would  make  on  the  basis  of  their  design 
experience  fit  into  one  of  several  categories:  (1)  a change  in  approach  or  strategy,  (2)  a 
change  in  the  relative  value  given  to  various  goals,  (3)  more  or  less  attention  to  particular 
details.  This  finding  suggests  that,  in  this  particular  domain  at  least,  a useful  design  aid  may 
simply  be  a checklist  based  on  this  limited  number  of  concerns.  This  would  "remind"  partici- 
pants at  the  outset  of  variables  they  might  not  otherwise  consider  except  in  retrospect. 


An  alternative  way  of  looking  at  answers  was  to  perform  a word  frequency  analysis  on  the 
answers  to  this  question.  The  frequently  occurring  words  could  be  classified  according  to:  1) 
financial  considerations  (expenses,  cost,  spend,  money),  2)  procedural  changes  (first,  different- 
ly, start,  approach,  how),  3)  quantitative  changes  (more,  much),  and  combination  or  complexi- 
ty words:  (and,  with,  in).  Again,  the  ways  that  participants  would  change  their  behavior 
include  strategy  changes,  changes  in  the  relative  importance  of  various  goals,  and  attention  to 
particular  details.  In  addition,  it  appears  that  some  participants  would  do  much  the  same 
things  but  in  a different  order  or  in  different  combinations. 
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Other  questions  concerning  strategy  revealed  that  the  majority  of  participants  claimed  to  ; 

have  worked  on  the  hard  parts  first  (19  out  of  29),  and  to  have  planned  out  their  approach 
before  beginning  (19  out  of  29).  Neither  of  these  answers  was  related  to  objective  perform- 
ance measures.  These  results  again  support  the  contention  that  participants  were  more 
systematic  in  their  approach  than  one  might  have  suspected  on  the  basis  of  the  folklore  of 
software  design. 


Units  of  design.  Participants  were  also  asked  essentially  what  their  units  of  design  were. 
The  results  were  that  8 said  areas;  4,  ideas;  4,  mixed;  2,  rooms;  and  one  participant  each 
claimed  the  following:  subrooms,  specific  features,  kitchen  first,  4’  by  4’  squares,  and  miscella- 
neous. A word  frequency  analysis  of  the  answers  to  this  question  turned  up  the  following  in 
order  from  most  to  least  frequent  — rooms,  areas,  (and),  (the),  ideas,  area,  tables.  Partici- 
pants seem  to  perceive  a variety  of  units  of  design  as  appropriate  even  in  this  fairly  simple 
design  problem.  A structured  approach  based  on  a pre-existing  set  of  units  might  be  difficult 
for  some  people  to  use,  though  this  hypothesis  requires  direct  test.  Also  suggested  by  the 
above  results  is  the  classification  of  alternate  sub-designs  by  multiple,  possibly  over-lapping 
units.  This  would  allow  people  who  would  spontaneously  analyse  the  problem  differently  to 
access  possibly  relevant  information. 

3.4  Differences  in  the  Modifiability  of  Existing  Structures 

Although  the  essence  of  design  is  that  something  new  is  created,  the  new  is  always  within 
the  context  of  some  pre-existing  structures,  procedures,  or  units.  In  the  present  problem, 
participants  were  free  to  choose  the  extent  to  which  they  were  willing  to  modify  the  pre- 
existing structure  of  the  church.  It  was  hypothesized  that  participants  would  be  most  likely  to 
change  furniture  and  architectural  details,  somewhat  less  likely  to  change  the  moveable 
partitions,  less  likely  to  change  internal  non-structural  walls,  less  likely  to  change  internal 
structural  walls,  and  least  likely  to  change  external  walls.  If  one  sums  across  participants,  this 
was  certainly  true.  Only  three  participants  changed  the  external  walls,  12  changed  the 
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structural  walls,  16  changed  the  non-structural  walls,  18  changed  the  partitions,  and  all  the 
participants  changed  the  furniture  or  architectural  detail.  With  regard  to  the  stronger  hypothe- 
sis that  such  changes  formed  a strict  hierarchy;  among  the  29  participants  there  was  no  case  of 
a reversal  of  this  hierarchy  (e.g.,  a participant  who  changed  structural  walls  but  left  non- 
structural  walls  alone).  To  do  so  would  have  been  physically  possible  but  participants  never 
violated  the  hierarchy. 

This  consistency  provides  some  indirect  support  for  the  notion  that  people  share  a 
consensus  about  the  levels  in  a hierarchy,  at  least  for  the  case  of  structural  design.  Such  a 
consensus  is  a desirable  feature  if  some  of  the  potential  benefits  of  communicability  accruing 
to  top-down  design  are  to  be  realized. 


There  have  appeared  some  experimental  investigations  of  hierarchies  in  the  literature  in 
cognitive  psychology.  Generally,  such  studies  have  been  more  concerned  with  what  the 
consensually  determined  hierarchy  is  rather  than  with  the  degree  of  consensus  though  there  are 
some  exceptions  (Durding,  Becker,  and  Gould,  1977;  Friendly,  1977). 

3.5  Dimensions  of  Style 

In  addition  to  looking  at  the  features  of  the  restaurant  designs  that  participants  included, 
the  designs  were  also  characterized  according  to  style  as  defined  below.  First,  participants 
could  communicate  their  design  choices  via  some  combination  of  a price  list,  a verbal  descrip- 
tion, a floor  plan,  or  pictorial  views  of  the  restaurant.  The  various  plans  were  characterized 
for  style  according  to  whether  the  participant:  1)  used  color  in  the  design,  2)  evidenced 
concern  with  three  dimensions,  3)  included  contextual  description  or  pictures,  and  4)  showed 
asymmetry  in  the  design.  These  particular  stylistic  variables  were  chosen  because  the  use  of 
color,  three-dimensions,  concreteness  and  scope,  and  a preference  for  asymmetry  have  all  been 
claimed  to  be  characteristic  of  creative  people  (Stein,  1974).  It  was  hypothesized  that  such 
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styles  would  be  found  among  those  participants  who  scored  relatively  high  on  originality  but 
not  practicality. 

With  respect  to  the  mode  of  communicating  ideas,  every  one  of  the  29  participants 
included  a floor  plan.  In  addition,  20  gave  verbal  descriptions,  17  gave  lists  of  costs,  and  7 
gave  one  or  more  pictorial  views  of  some  aspect  of  the  design.  These  means  of  communication 
occurred  in  many  possible  combinations. 

Twelve  participants  used  only  one  color  in  the  design  plan,  nine  used  two  colors,  six  used 
three  colors,  and  two  used  five  colors.  Only  four  of  the  participants  gave  explicit  evidence  of 
concern  with  the  dimension  of  height.  Seven  participants  gave  diagrams  that  showed  the 
surrounding  context  of  the  restaurant.  Five  participants  showed  symmetry  in  the  overall  design 
and  15  had  seating  primarily  in  rows. 

These  style  variables  were  not,  however,  correlated  with  practicality  or  originality.  As 
noted,  it  was  expected  that  a correlation  with  originality  would  occur.  The  independence  of 
stylistic  features  from  the  objective  measures  of  originality  and  practicality  may  suggest  that 
the  act  of  communicating  about  the  design  is  fairly  separable  in  this  case  from  the  actual 
creation. 

An  incidental  observation  of  potential  importance  to  education  is  the  extent  to  which 
these  participants  became  involved  in  the  design  of  the  restaurant.  This  was  evidenced  by  the 
fact  that  though  the  participants  were  all  told  that  they  would  be  paid  for  four  hours  regardless 
of  the  time  they  actually  spent,  only  one  participant  attempted  to  hand  in  a design  before  three 
hours  and  twenty  minutes.  Giving  complex  design  problems  that  require  the  application  of 
knowledge  is  probably  a very  effective  manner  of  teaching  the  knowledge.  Educational 
research  on  this  point  may  prove  very  fruitful. 
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4.  Discussion. 
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4. 1 Goals  of  Design. 

Previous  research  (e.g..  White,  Terence,  & Cecil,  1977;  Terborg  & Wilier,  1978)  indicates 
that  explicit  goal-setting  influences  performance.  Although  typically,  performance  has  been 
measured  uni-dimensionally,  Terborg  & Miller  (1978)  took  several  measures  of  performance. 

The  current  experiment  extends  previous  research  to  more  creative  cognitive  activity  and 
proposes  two  important  dimensions  of  goals  and  performance.  We  found  that  the  participant's 
assessment  of  where  he  or  she  stood  on  the  tradeoff  between  practicality  and  originality  was 
encouragingly  consistent.  The  participants  tended  consistently  to  fall  into  two  groups;  those 
who  stressed  practicality  and  workability  on  the  one  hand,  and  those  who  stressed  originality, 
novelty,  and  imagination  on  the  other.  Furthermore,  there  was  some  relationship  of  these 
questionnaire  results  to  the  objective  measures  of  O and  P.  A participant  of  a given  ability 
level  seems  to  be  able  to  control,  or  at  least  report  on,  the  extent  to  which  he  or  she  will  strive 
for  originality  at  the  expense  of  practicality  or  vice  versa.  If  participants  can  tradeoff  in  this 
manner  between  originality  and  practicality  more  accurate  assessments  of  an  individual’s 
abilities  may  be  made  if  both  factors  are  taken  into  account.  Also,  clients  of  designers  should 
make  more  explicit  their  own  desired  tradeoff  between  originality  and  practicality.  Extensions 
of  the  study  of  design  problem  solving  should  examine  quantitative  and  qualitative  goal-setting 
as  independent  variables. 

4.2  Aids  to  Idea  Production. 

The  fact  that  participants  often  failed  to  meet  all  of  the  functional  requirements  that  we 
considered  important  for  a restaurant  suggests  that  a simple  checklist  of  such  requirements 
would  have  improved  the  practicality  of  the  designs  considerably.  Even  the  presence  of  an 
unrelated  list  of  words  apparently  acted  as  a cue  to  consider  more  possibilities.  Collaboration 
can  be  thought  of  as  another  'aid'  to  the  design  process  in  that  two  participants  will  probably 
consider  more  ideas  than  one  participant.  However,  the  literature  on  group  problem  solving 
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indicates  that  the  value  of  collaboration  will  depend  upon  where  in  the  problem  solving  process 
collaboration  is  introduced  as  well  as  the  mechanisms  for  interacting  (See  Stein,  1975  for  a 
review  of  group  procedures  for  improving  problem  solving). 
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4. 3 Measures  of  the  Design  Product. 

There  were  significant  correlations  between  the  theory-based  measures  of  originality  and 
practicality  and  independent  variables.  Further  development  of  these  measures  could  provide 
an  important  tool  for  studying  complex  problem  solving.  An  alternative  procedure,  explored  in 
a number  of  more  recent  experiments  (Carroll,  Thomas,  and  Malhotra,  1977;  Carroll,  Thomas, 
Miller,  and  Malhotra,  1977)  is  to  structure  a design  problem  so  that  more  objective  measures 
are  feasible. 
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Figure  Captions. 


Figure  1.  The  layout  of  the  church  in  which  e participants  were  to  design  a restaurant. 


Figure  2.  Goal  tree  for  a restaurant  design. 


Figure  3.  The  distribution  of  objective  originality  scores. 


Figure  4.  The  distribution  of  objective  practicality  scores. 
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